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DETAILED ACTION 
Remarks 

1 . Receipt of Applicant's Amendment, filed on 08/05/201 0, is acknowledged. The 
amendment includes the amending of claims 1, 6-8, and 11, and the cancellation of 
claims 2-3, and 18. 

Priority 

2. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), which 
papers have been placed of record in the file. 

Claim Rejections - 35 USC § 101 

3. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, macliine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

4. Independent claim 7 is directed towards a viewer program. However, all of the 
elements claimed could be reasonably interpreted by an ordinary artisan as being 
software alone, and thus is directed to software per se, which is non-statutory. 

In order for such a software claim to be statutory, it must be claimed in 
combination with an appropriate medium and/or hardware such as a memory or 
processor to establish a statutory category of invention and enable any functionality to 
realized. 

Specifically, there is no explicit limitation in the body of independent claim 7 that 
states that the viewer program is stored and executed on an appropriate medium. 
Rather, all that is stated is that the viewer program causes a computer to perform 
operations. Moreover, the claimed computer in the preamble of the independent claim 
is insufficient as it is merely used in attempt at intended use. 

Claim 16 is rejected for incorporating the deficiencies of independent claim 7. 
Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
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the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1 , 4, 6-8, 1 1 , and 1 3-1 7 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Hyon (U.S. PGPUB 2002/0077135) in view of Baker (U.S. Patent 
6,546,417), and further in view of Ostermann et al. (U.S. Patent 6,990,452). 

7. Regarding claim 1 , Hyon teaches a display device comprising: 

A) a first storage portion for storing a character code for specifying said predetermined 
kind of character, and character shape data corresponding to said character code in a 
correlated manager (Paragraphs 23, and 41-42, Figure 4); 

B) a second storage portion for storing an image code for specifying said registered 
image and registered image data corresponding to said image code in a correlated 
manner according to registration processing by a user (Paragraphs 23, 25, and 44, 
Figure 2); 

C) a display output portion for outputting said predetermined kind of character and said 
registered image (Paragraph 21 , Figure 2); 

D) a display control portion for causing said display output portion to output 
corresponding said predetermined king of character and said registered image 
(Paragraphs 21 and 41-42); 

E) wherein at least one predetermined kind of character is to be displayed directly 
adjacent to said registered image in accordance with display data containing said 
character code, text attribute data, and said image code (Paragraphs 21 and 41-42). 

The examiner notes that Hyon teaches "a first storage portion for storing a 
character code for specifying said predetermined l^ind of character, and character 
shape data corresponding to said character code in a correlated manager" as "A 
storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
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(Paragraph 23) and "the user can input a text including typical characters, specical 
characters, or emoticons within the range of a transmittable SMS message, by changing 
input modes" (Paragraph 41 ). The examiner further notes that Hyon teaches "a 
second storage portion for storing an image code for specifying said registered 
image and registered image data corresponding to said image code in a 
correlated manner according to registration processing by a user" as "A storage 
18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access Memory) 
for storing programs and data, and a voice memory. The storage 18 stores an operation 
program for inputting emoticons and a plurality of emoticons in the form of a bit map 
according to the embodiment of the present invention. The plurality of emoticons are 
stored by groups in the storage 18 to facilitate selection of emoticons" (Paragraph 23), 
"storage 18 stores oriental or occidental emoticons according to the cultural area of a 
user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a mobile terminal 
so that a user easily select an intended emoticon in the present invention. In another 
embodiment, the user can add hieroglyphics to basic emoticons stored by a 
manufacturer. The hieroglyphics can be stored in three ways: the user directly stores 
them, receives them from a base station and the stores them, or downloads them from 
the Internet. The user can change and edit the emoticons stored in the mobile terminal" 
(Paragraph 44). The examiner further notes that it is clear that the emoticons which are 
edited, changed, and user-generated must be stored in the RAM of storage 18 since 
items stored in RAM can be written to whereas items stored in ROM cannot be written 
(i.e. are Read-Only). The examiner further notes that items which are clearly not 
edited/changed/manipulated (i.e. text characters) would be stored in ROM of storage 
18. Moreover, the examiner further wishes to state that it is well known that text 
characters from a particular font are stored in ROM and not RAM (see cited reference of 
Hersum (see column 4, lines 55-67 for reference). The examiner further notes that 
Hyon teaches "a display output portion for outputting said predetermined kind of 
character and said registered image" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
used as the display 1 6" (Paragraph 21 ). The examiner further notes that Hyon teaches 
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"a display control portion for causing said display output portion to output 
corresponding said predetermined king of character and said registered image" 

as "a display 16 outputs display data and text messages generated in the mobile 
terminal. An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 
21), "If the user selects "smile" in the display 6 by manipulating a left/right directional 
key, emoticons that belong to the group "smile" are displayed in a display 7. When the 
user chooses one of the "smile" emoticons " . " in the display 7, the selected emoticon is 
stored as part of the short message as shown in a display 8. Then, the user can input a 
text including typical characters, special characters, or emoticons within the range of a 
transmittable SMS message, by changing input modes" (Paragraph 41), and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon... the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that Hyon teaches " wherein at least one 
predetermined kind of character is to be displayed directly adjacent to said 
registered image in accordance with display data containing said character code, text 
attribute data, and said image code" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
used as the display 16" (Paragraph 21), "If the user selects "smile" in the display 6 by 
manipulating a left/right directional key, emoticons that belong to the group "smile" are 
displayed in a display 7. When the user chooses one of the "smile" emoticons " . " in the 
display 7, the selected emoticon is stored as part of the short message as shown in a 
display 8. Then, the user can input a text including typical characters, special 
characters, or emoticons within the range of a transmittable SMS message, by changing 
input modes" (Paragraph 41), and "After an intended text is completed, the user 
instructs the mobile terminal to transmit the SMS message including the emoticon... the 
mobile terminal transmits the stored message" (Paragraph 42). The examiner further 
notes that it is common knowledge that when text messages are received via a device, 
the entire message is displayed at once. 
Hyon does not explicitly teach: 
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F) said text attribute data including a size mark to be added to each said at least one 
directly adjacent character ; 

G) said display control portion determined whether or not a size of said at least one 
directly adjacent character and a size of the registered image are substantially the 

same; 

H) when the size of said at least one directly adjacent character and a size of the 
registered image are substantially the same, when the size of at least one directly 
adjacent character and the size of the registered image are not substantially the same, 
said display control portion makes the size of the at least one directly adjacent character 
and the size of the registered image substantially the same by scaling up/down the size 
of the registered image based on the text attribute data; 

J) said display control portions causes said display output portion to display a text 
containing said at least one directly adjacent character and the registered image scaled 
to substantially the same size . 

Baker, howeyer, teaches "said text attribute data including a size marl< to be 
added to each said at least one directly adiacent character " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also proyided...at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "said 
display control portion determined whether or not a size of said at least one 
directly adiacent character and a size of the registered image are substantially the 
same" as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
image for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), "when the size of said at least one directly adiacent character and a size of 
the registered image are substantially the same, when the size of at least one 
directly adiacent character and the size of the registered image are not 
substantially the same, said display control portion makes the size of the at least 
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one directly adjacent character and the size of the registered image substantially 
the same by scaling up/down the size of the registered image based on the text 
attribute data" as "in order to accommodate the use of different size fonts in the 
mailbox display, means for scaling the size of the icon graphics are also provided... at 
least one image for each icon is stored, the icon most closely matching the point size of 
the font is chosen and then scaled as needed to better match the font point size" 
(Column 8, lines 26-40), and "said display control portions causes said display 
output portion to display a text containing said at least one directly adjacent 
character and the registered image scaled to substantially the same size " as "in 
order to accommodate the use of different size fonts in the mailbox display, means for 
scaling the size of the icon graphics are also provided... at least one image for each icon 
is stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
F) said text attribute data including a style mark to be added to each said at least one 
directly adiacent character : 

I) said display control portion modifies said registered image to include image data for a 
mark resembling said style make attribute data of said at least one directly adiacent 

character : 

J) and including the mark resembling the style mark attribute data in a position that is 
substantially continuous with the placement of the style mark in said at least one directly 
adiacent character . 

Ostermann, however, teaches "said text attribute data including a style mark 
to be added to each said at least one directly adiacent character " as "The sender 
may also insert emoticons 103 into the text of the message. The system includes 
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predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye 
wink, and so forth. The predefined emoticons are represented either as icons or as text, 
such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of emoticon 
icons 96. The sender inserts an emoticon into a text message at the location of the 
cursor 102 by clicking on one of the emoticon icons 100. The sender may also type in 
the desired emoticon as text. Emoticon icons 96 save the sender from needing to type 
three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture 
of, say, a winking eye or a icon representation of the characters ";-)" 100, or other 
information indicating to the sender that clicking on that emoticon icon will insert the 
associated emotion 103 into the text at the location of the cursor 102" (Column 8, lines 
35-50) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color. In this case, in 
a sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ), "said 
display control portion modifies said registered image to include image data for a 
mark resembling said style make attribute data of said at least one directly 
adjacent character " as "The sender may also insert emoticons 103 into the text of the 
message. The system includes predefined emoticons 96, such as ":-) " for a smile, "::-)" 
for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)". As shown in FIG. 6, the window 80 
includes a sample group of emoticon icons 96. The sender inserts an emoticon into a 
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text message at the location of tlie cursor 102 by eliciting on one of tine emoticon icons 
1 00. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
Icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" ICQ, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color. In this case, in a sentence such as "Hi John, :-) are you pleased 
that the stock market is up?" the underlining represents the highlighting wherein the 
sender chooses to begin the smile at the beginning of the word "are" and to continue 
the smile through the word "up". The method comprises receiving the indicated duration 
of the emotion and presenting the chosen duration of the emotion as the animated entity 
delivers the message. As mentioned above, the highlighting can occur through coloring 
words, underlining words, or some other means of presenting the emotion" (Column 1 1 , 
lines 43-61), and " and including the mark resembling the style mark attribute data 
in a position that is substantially continuous with the placement of the style mark 
in said at least one directly adiacent character " as "The sender may also insert 
emoticons 103 into the text of the message. The system includes predefined emoticons 
96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. 
The predefined emoticons are represented either as icons or as text, such as ";-)". As 
shown in FIG. 6, the window 80 includes a sample group of emoticon icons 96. The 
sender inserts an emoticon into a text message at the location of the cursor 102 by 
clicking on one of the emoticon icons 100. The sender may also type in the desired 
emoticon as text. Emoticon icons 96 save the sender from needing to type three keys, 
such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture of, say, a 
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winking eye or a icon representation of tlie cliaracters ";-)" 100, or other information 
indicating to the sender that eliciting on that emoticon icon will insert the associated 
emotion 103 into the text at the location of the cursor 102" (Column 8, lines 35-50) and 
"The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
the sender an option to assign a color to the at least typed one word such that the 
chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color. In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61). 

The examiner further notes that Figure 6 of Ostermann displays a selectable 
area 96 for users to insert emoticons directly via a mouse into a currently typed 
message. Moreover, Ostermann explicitly states that the style markings of an adjacent 
character are applied to an emoticon (See "the sender an option to assign a color to the 
at least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color") 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 



Application/Control Number: 10/516,881 
Art Unit: 2168 



Page 1 1 



Regarding claim 4, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein when said registered is a gray image, said display control portion converts 
each pixel of said registered image into a color made by mixing the fore color and a 
back color of said text at a ratio according to a pixel value of said predetermined kind of 

character. 

Ostermann, however, teaches "wherein when said registered is a gray 
image, said display control portion converts each pixel of said registered image 
into a color made by mixing the fore color and a back color of said text at a ratio 
according to a pixel value of said predetermined kind of character" as "The 
increased intensity of the emoticon may be accomplished by changing the icon from 
black-on-white background to black-on-colored background (such as red or yellow) 
where the intensity of the background color reflects the amplitude" (Column 9, lines 43- 
67) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color" (Column 1 1 , 
lines 43-50) 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 5, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said text attribute data contains decoration attribute data indicating a type of 
a decoration applied to a corresponding predetermined kind of character; and 
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B) said display control portion decorates said registered image according to said 
decoration attribute data. 

Ostermann, however, teaches "wherein said text attribute data contains 
decoration attribute data indicating a type of a decoration applied to a 
corresponding predetermined kind of character" as "The sender can associate 
typed words with an emoticon by underlining, coloring, highlighting, or by any other 
means... In this case, in a sentence such as "Hi John, :-) are you pleased that the stock 
market is up?" the underlining represents the highlighting wherein the sender chooses 
to begin the smile at the beginning of the word "are" and to continue the smile through 
the word "up". The method comprises receiving the indicated duration of the emoticon 
and presenting the chosen duration of the emoticon" (Column 1 1 , lines 43-61) and 
"said display control portion decorates said registered image according to said 
decoration attribute data" as "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
chosen duration of the emoticon" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 6, Hyon teaches a display method comprising: 
A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Paragraphs 23, 25, and 44, Figure 2); 
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B) said registered image stored in a storage portion different from a storage portion of 
storing the predetermined kind of character (Paragraphs 23 and 44); 

C) at least one predetermined l<ind of character that is to be displayed directly adjacent 
to said registered image with respect to display data containing a character code, text 
attribute data, and said image code (Paragraphs 21 and 41-42); 

F) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42); 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing by a user" as "A 

storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23), "storage 18 stores oriental or occidental emoticons according to the 
cultural area of a user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a 
mobile terminal so that a user easily select an intended emoticon in the present 
invention. In another embodiment, the user can add hieroglyphics to basic emoticons 
stored by a manufacturer. The hieroglyphics can be stored in three ways: the user 
directly stores them, receives them from a base station and the stores them, or 
downloads them from the Internet. The user can change and edit the emoticons stored 
in the mobile terminal" (Paragraph 44). The examiner further notes that it is clear that 
the emoticons which are edited, changed, and user-generated must be stored in the 
RAM of storage 1 8 since items stored in RAM can be written to whereas items stored in 
ROM cannot be written (i.e. are Read-Only). The examiner further notes that items 
which are clearly not edited/changed/manipulated (i.e. text characters) would be stored 
in ROM of storage 1 8. Moreover, the examiner further wishes to state that it is well 
known that text characters from a particular font are stored in ROM and not RAM (see 
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cited reference of Hersum (see column 4, lines 55-67 for reference). The examiner 
further notes that Hyon teaches "said registered image stored in a storage portion 
different from a storage portion of storing the predetermined kind of character" 

as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random 
Access Memory) for storing programs and data, and a voice memory. The storage 1 8 
stores an operation program for inputting emoticons and a plurality of emoticons in the 
form of a bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23) and "In another embodiment, the user can add hieroglyphics to basic 
emoticons stored by a manufacturer. The hieroglyphics can be stored in three ways: 
the user directly stores them, receives them from a base station, and the stores them, or 
downloads them from the Internet" (Paragraph 44). The examiner further notes that 
Hyon teaches " at least one predetermined l<ind of character that is to be displayed 
directly adiacent to said registered image with respect to display data containing 
a character code, text attribute data, and said image code" as "a display 16 outputs 
display data and text messages generated in the mobile terminal. An LCD (Liquid 
Crystal Display) can be used as the display 16" (Paragraph 21), "If the user selects 
"smile" in the display 6 by manipulating a left/right directional key, emoticons that belong 
to the group "smile" are displayed in a display 7. When the user chooses one of the 
"smile" emoticons " . " in the display 7, the selected emoticon is stored as part of the 
short message as shown in a display 8. Then, the user can input a text including typical 
characters, special characters, or emoticons within the range of a transmittable SMS 
message, by changing input modes" (Paragraph 41), and "After an intended text is 
completed, the user instructs the mobile terminal to transmit the SMS message 
including the emoticon... the mobile terminal transmits the stored message" (Paragraph 
42). The examiner further notes that Hyon teaches "displaying said predetermined 
kind of character and said registered image simultaneously based on said 
transformed registered image" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
used as the display 16" (Paragraph 21), "If the user selects "smile" in the display 6 by 
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manipulating a left/right directional key, emoticons that belong to the group "smile" are 
displayed in a display 7. When the user chooses one of the "smile" emoticons " . " in the 
display 7, the selected emoticon is stored as part of the short message as shown in a 
display 8. Then, the user can input a text including typical characters, special 
characters, or emoticons within the range of a transmittable SMS message, by changing 
input modes" (Paragraph 41 ), and "After an intended text is completed, the user 
instructs the mobile terminal to transmit the SMS message including the emoticon... the 
mobile terminal transmits the stored message" (Paragraph 42). The examiner further 
notes that it is common knowledge that when text messages are received via a device, 
the entire message is displayed at once. 
Hyon does not explicitly teach: 

D) said text attribute data containing a size mark to be added to each said at least one 
directiv adjacent character : 

E) transforming said registered image to be displayed according to said text attribute 
data; 

H) by determining whether or not a size of said at least one directiv adjacent character 
and a size of the registered image are substantially the same, when the size of at least 
one directiv adjacent character and the size of the registered image are not substantially 
the same, said display control portion makes the size of the at least one directiv 
adjacent character and the size of the registered image substantially the same by 
scaling up/down the size of the registered image based on the text attribute data; 
J) causing said display output portion to display a text containing said at least one 
directiv adjacent character and the registered image scaled to substantiallv the same 
size. 

Baker, however, teaches "said text attribute data containing a size mark to 
be added to each said at least one directiv adjacent character " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), 
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"transforming said registered image to be displayed according to said text 
attribute data" as "in order to accommodate the use of different size fonts in the 
mailbox display, means for scaling the size of the icon graphics are also provided... at 
least one image for each icon is stored, the icon most closely matching the point size of 
the font is chosen and then scaled as needed to better match the font point size" 
(Column 8, lines 26-40), "by determining whether or not a size of said at least one 
directly adjacent character and a size of the registered image are substantially the 
same, when the size of at least one directly adjacent character and the size of the 
registered image are not substantially the same, said display control portion 
makes the size of the at least one directly adjacent character and the size of the 
registered image substantially the same by scaling up/down the size of the 
registered image based on the text attribute data" as "in order to accommodate the 
use of different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and "causing said display 
output portion to display a text containing said at least one directly adjacent 
character and the registered image scaled to substantially the same size " as "in 
order to accommodate the use of different size fonts in the mailbox display, means for 
scaling the size of the icon graphics are also provided. ..at least one image for each icon 
is stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
D) said text attribute data containing a stvie mark to be added to each said at least one 
directly adjacent character : 
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G) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line; 

I) modifying said registered image to include image data for a mark resembling said 
style make attribute data of said at least one directly adjacent character : 

J) and including the mark resembling the style mark attribute data in a position that is 
substantially continuous with the placement of the style mark in said at least one directly 
adjacent character . 

Ostermann, howeyer, teaches "said text attribute data containing a style 
marl< to be added to each said at least one directly adjacent character " as "The 
sender may also insert emoticons 103 into the text of the message. The system 
includes predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" 
for an eye wink, and so forth. The predefined emoticons are represented either as icons 
or as text, such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of 
emoticon icons 96. The sender inserts an emoticon into a text message at the location 
of the cursor 102 by clicking on one of the emoticon icons 100. The sender may also 
type in the desired emoticon as text. Emoticon icons 96 saye the sender from needing 
to type three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a 
picture of, say, a winking eye or a icon representation of the characters ";-)" 100, or 
other information indicating to the sender that clicking on that emoticon icon will insert 
the associated emotion 103 into the text at the location of the cursor 102" (Column 8, 
lines 35-50) and "The sender can associate typed words with an emoticon by 
underlining, coloring, highlighting, or by any other means. For example, the method 
may comprise proyiding the sender an option to assign a color to the at least typed one 
word such that the chosen emoticon begins to be presented by the animated entity to 
the recipient at the first typed word with the assigned color and the chosen emoticon 
presentation by the animated entity ends at the last typed word with the assigned color. 
In this case, in a sentence such as "Hi John, :-) are you pleased that the stock market is 
up?" the underlining represents the highlighting wherein the sender chooses to begin 
the smile at the beginning of the word "are" and to continue the smile through the word 
"up". The method comprises receiying the indicated duration of the emotion and 
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presenting the chosen duration of the emotion as the animated entity delivers the 
message. As mentioned above, the highlighting can occur through coloring words, 
underlining words, or some other means of presenting the emotion" (Column 1 1 , lines 
43-61), "so that said registered image is sandwiched between two parts of said 
text of said predetermined kind of character in a line" as "The sender may also 
insert emoticons 103 into the text of the message. The system includes predefined 
emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye wink, and 
so forth. The predefined emoticons are represented either as icons or as text, such as 
";-)". As shown in FIG. 6, the window 80 includes a sample group of emoticon icons 96. 
The sender inserts an emoticon into a text message at the location of the cursor 102 by 
clicking on one of the emoticon icons 100. The sender may also type in the desired 
emoticon as text. Emoticon icons 96 save the sender from needing to type three keys, 
such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture of, say, a 
winking eye or a icon representation of the characters ";-)" 100, or other information 
indicating to the sender that clicking on that emoticon icon will insert the associated 
emotion 103 into the text at the location of the cursor 102" (Column 8, lines 35-50) and 
"The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
the sender an option to assign a color to the at least typed one word such that the 
chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color. In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61), " modifying 
said registered image to include image data for a mark resembling said style 
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make attribute data of said at least one directly adjacent character " as "The sender 
may also insert emoticons 103 into the text of the message. The system includes 
predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye 
wink, and so forth. The predefined emoticons are represented either as icons or as text, 
such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of emotlcon 
Icons 96. The sender Inserts an emotlcon into a text message at the location of the 
cursor 102 by clicking on one of the emotlcon icons 100. The sender may also type In 
the desired emotlcon as text. Emotlcon icons 96 save the sender from needing to type 
three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture 
of, say, a winking eye or a icon representation of the characters ";-)" 100, or other 
information indicating to the sender that clicking on that emotlcon icon will insert the 
associated emotion 103 Into the text at the location of the cursor 102" (Column 8, lines 
35-50) and "The sender can associate typed words with an emotlcon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emotlcon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emotlcon presentation by 
the animated entity ends at the last typed word with the assigned color. In this case, In 
a sentence such as "HI John, :-) are you pleased that the stock market Is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ), and " and 
including the mark resembling the style mark attribute data in a position that is 
substantially continuous with the placement of the style mark in said at least one 
directly adjacent character " as "The sender may also insert emoticons 103 into the 
text of the message. The system includes predefined emoticons 96, such as ":-) " for a 
smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined 
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emoticons are represented either as icons or as text, sucli as ";-)". As sliown in FIG. 6, 
the window 80 includes a sample group of emoticon icons 96. The sender inserts an 
emoticon into a text message at the location of the cursor 102 by clicking on one of the 
emoticon icons 100. The sender may also type in the desired emoticon as text. 
Emoticon icons 96 save the sender from needing to type three keys, such as ":" and "-" 
and ")" for a smile. The icons 96 may be either a picture of, say, a winking eye or a icon 
representation of the characters ";-)" 100, or other information indicating to the sender 
that clicking on that emoticon icon will insert the associated emotion 103 into the text at 
the location of the cursor 102" (Column 8, lines 35-50) and "The sender can associate 
typed words with an emoticon by underlining, coloring, highlighting, or by any other 
means. For example, the method may comprise providing the sender an option to 
assign a color to the at least typed one word such that the chosen emoticon begins to 
be presented by the animated entity to the recipient at the first typed word with the 
assigned color and the chosen emoticon presentation by the animated entity ends at the 
last typed word with the assigned color. In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
continue the smile through the word "up". The method comprises receiving the indicated 
duration of the emotion and presenting the chosen duration of the emotion as the 
animated entity delivers the message. As mentioned above, the highlighting can occur 
through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

The examiner further notes that Figure 6 of Ostermann displays a selectable 
area 96 for users to insert emoticons directly via a mouse into a currently typed 
message. Moreover, Ostermann explicitly states that the style markings of an adjacent 
character are applied to an emoticon (See "the sender an option to assign a color to the 
at least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color") 
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It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 7, Hyon teaches a viewer program comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Paragraphs 23, 25, and 44, Figure 2); 

B) said registered image stored in a storage portion different from a storage portion of 
storing the predetermined kind of character (Paragraphs 23 and 44); 

C) at least one predetermined kind of character that is to be displayed directly adjacent 
to said registered image with respect to display data containing a character code, text 
attribute data, and said image code (Paragraphs 21 and 41-42); 

F) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42); 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing by a user" as "A 

storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 

operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23), "storage 18 stores oriental or occidental emoticons according to the 
cultural area of a user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a 
mobile terminal so that a user easily select an intended emoticon in the present 
invention. In another embodiment, the user can add hieroglyphics to basic emoticons 
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stored by a manufacturer. The hieroglyphics can be stored in three ways: the user 
directly stores them, receives them from a base station and the stores them, or 
downloads them from the Internet. The user can change and edit the emoticons stored 
in the mobile terminal" (Paragraph 44). The examiner further notes that it is clear that 
the emoticons which are edited, changed, and user-generated must be stored in the 
RAM of storage 18 since items stored in RAM can be written to whereas items stored in 
ROM cannot be written (i.e. are Read-Only). The examiner further notes that items 
which are clearly not edited/changed/manipulated (i.e. text characters) would be stored 
in ROM of storage 1 8. Moreover, the examiner further wishes to state that it is well 
known that text characters from a particular font are stored in ROM and not RAM (see 
cited reference of Hersum (see column 4, lines 55-67 for reference). The examiner 
further notes that Hyon teaches "said registered image stored in a storage portion 
different from a storage portion of storing the predetermined kind of character" 
as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random 
Access Memory) for storing programs and data, and a voice memory. The storage 18 
stores an operation program for inputting emoticons and a plurality of emoticons in the 
form of a bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23) and "In another embodiment, the user can add hieroglyphics to basic 
emoticons stored by a manufacturer. The hieroglyphics can be stored in three ways: 
the user directly stores them, receives them from a base station, and the stores them, or 
downloads them from the Internet" (Paragraph 44). The examiner further notes that 
Hyon teaches " at least one predetermined kind of character that is to be displayed 
directiv adjacent to said registered image with respect to display data containing 
a character code, text attribute data, and said image code" as "a display 16 outputs 
display data and text messages generated in the mobile terminal. An LCD (Liquid 
Crystal Display) can be used as the display 16" (Paragraph 21), "If the user selects 
"smile" in the display 6 by manipulating a left/right directional key, emoticons that belong 
to the group "smile" are displayed in a display 7. When the user chooses one of the 
"smile" emoticons " . " in the display 7, the selected emoticon is stored as part of the 
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short message as shown in a display 8. Then, the user can input a text including typical 
characters, special characters, or emoticons within the range of a transmittable SMS 
message, by changing input modes" (Paragraph 41), and "After an intended text is 
completed, the user instructs the mobile terminal to transmit the SMS message 
including the emoticon...the mobile terminal transmits the stored message" (Paragraph 
42). The examiner further notes that Hyon teaches "displaying said predetermined 
kind of cliaracter and said registered image simultaneously based on said 
transformed registered image" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
used as the display 16" (Paragraph 21), "If the user selects "smile" in the display 6 by 
manipulating a left/right directional key, emoticons that belong to the group "smile" are 
displayed in a display 7. When the user chooses one of the "smile" emoticons " . " in the 
display 7, the selected emoticon is stored as part of the short message as shown in a 
display 8. Then, the user can input a text including typical characters, special 
characters, or emoticons within the range of a transmittable SMS message, by changing 
input modes" (Paragraph 41), and "After an intended text is completed, the user 
instructs the mobile terminal to transmit the SMS message including the emoticon... the 
mobile terminal transmits the stored message" (Paragraph 42). The examiner further 
notes that it is common knowledge that when text messages are received via a device, 
the entire message is displayed at once. 
Hyon does not explicitly teach: 

D) said text attribute data containing a size mark to be added to each said at least one 
directiv adiacent character : 

E) transforming said registered image to be displayed according to said text attribute 

data; 

H) by determining whether or not a size of said at least one directiv adjacent character 
and a size of the registered image are substantially the same, when the size of at least 
one directiv adiacent character and the size of the registered image are not substantially 
the same, said display control portion makes the size of the at least one directiv 
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adjacent character and the size of the registered image substantially the same by 
scaling up/down the size of the registered image based on the text attribute data; 
J) causing said display output portion to display a text containing said at least one 
directly adiacent character and the registered image scaled to substantially the same 
size . 

Baker, however, teaches "said text attribute data containing a size mark to 
be added to eacli said at least one directly adiacent character " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also proyided...at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), 
"transforming said registered image to be displayed according to said text 
attribute data" as "in order to accommodate the use of different size fonts in the 
mailbox display, means for scaling the size of the icon graphics are also proyided...at 
least one image for each icon is stored, the icon most closely matching the point size of 
the font is chosen and then scaled as needed to better match the font point size" 
(Column 8, lines 26-40), "by determining whether or not a size of said at least one 
directly adiacent character and a size of the registered image are substantially the 
same, when the size of at least one directly adiacent character and the size of the 
registered image are not substantially the same, said display control portion 
makes the size of the at least one directly adiacent character and the size of the 
registered image substantially the same by scaling up/down the size of the 
registered image based on the text attribute data" as "in order to accommodate the 
use of different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and "causing said display 
output portion to display a text containing said at least one directly adiacent 
character and the registered image scaled to substantially the same size " as "in 
order to accommodate the use of different size fonts in the mailbox display, means for 
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scaling the size of the icon graphics are also provided... at least one image for each icon 
is stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
D) said text attribute data containing a style mark to be added to each said at least one 
directly adjacent character : 

G) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line; 

I) modifying said registered image to include image data for a mark resembling said 
style make attribute data of said at least one directly adjacent character : 
J) and including the mark resembling the style mark attribute data in a position that is 
substantially continuous with the placement of the style mark in said at least one directly 
adjacent character . 

Ostermann, however, teaches "said text attribute data containing a style 
mark to be added to each said at least one directly adjacent character " as "The 
sender may also insert emoticons 103 into the text of the message. The system 
includes predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" 
for an eye wink, and so forth. The predefined emoticons are represented either as icons 
or as text, such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of 
emoticon icons 96. The sender inserts an emoticon into a text message at the location 
of the cursor 102 by clicking on one of the emoticon icons 100. The sender may also 
type in the desired emoticon as text. Emoticon icons 96 save the sender from needing 
to type three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a 
picture of, say, a winking eye or a icon representation of the characters ";-)" 100, or 
other information indicating to the sender that clicking on that emoticon icon will insert 
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the associated emotion 103 into the text at the location of the cursor 102" (Column 8, 
lines 35-50) and "The sender can associate typed words with an emoticon by 
underlining, coloring, highlighting, or by any other means. For example, the method 
may comprise providing the sender an option to assign a color to the at least typed one 
word such that the chosen emoticon begins to be presented by the animated entity to 
the recipient at the first typed word with the assigned color and the chosen emoticon 
presentation by the animated entity ends at the last typed word with the assigned color. 
In this case, in a sentence such as "Hi John, :-) are you pleased that the stock market is 
up?" the underlining represents the highlighting wherein the sender chooses to begin 
the smile at the beginning of the word "are" and to continue the smile through the word 
"up". The method comprises receiving the indicated duration of the emotion and 
presenting the chosen duration of the emotion as the animated entity delivers the 
message. As mentioned above, the highlighting can occur through coloring words, 
underlining words, or some other means of presenting the emotion" (Column 1 1 , lines 
43-61 ), "so that said registered image is sandwiched between two parts of said 
text of said predetermined kind of character in a line" as "The sender may also 
insert emoticons 103 into the text of the message. The system includes predefined 
emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye wink, and 
so forth. The predefined emoticons are represented either as icons or as text, such as 
";-)". As shown in FIG. 6, the window 80 includes a sample group of emoticon icons 96. 
The sender inserts an emoticon into a text message at the location of the cursor 102 by 
clicking on one of the emoticon icons 100. The sender may also type in the desired 
emoticon as text. Emoticon icons 96 save the sender from needing to type three keys, 
such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture of, say, a 
winking eye or a icon representation of the characters ";-)" 100, or other information 
indicating to the sender that clicking on that emoticon icon will insert the associated 
emotion 103 into the text at the location of the cursor 102" (Column 8, lines 35-50) and 
"The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
the sender an option to assign a color to the at least typed one word such that the 
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chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color. In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ), " modifying 
said registered image to include image data for a mark resembling said style 
make attribute data of said at least one directly adjacent character " as "The sender 
may also insert emoticons 103 into the text of the message. The system includes 
predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye 
wink, and so forth. The predefined emoticons are represented either as icons or as text, 
such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of emoticon 
icons 96. The sender inserts an emoticon into a text message at the location of the 
cursor 102 by clicking on one of the emoticon icons 100. The sender may also type in 
the desired emoticon as text. Emoticon icons 96 save the sender from needing to type 
three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture 
of, say, a winking eye or a icon representation of the characters ";-)" 100, or other 
information indicating to the sender that clicking on that emoticon icon will insert the 
associated emotion 103 into the text at the location of the cursor 102" (Column 8, lines 
35-50) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color. In this case, in 
a sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
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underlining represents the liigliligliting wlierein tlie sender cliooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ), and "and 
including the mark resembling the style mark attribute data in a position that is 
substantially continuous with the placement of the style mark In said at least one 
directly adjacent character " as "The sender may also insert emoticons 103 into the 
text of the message. The system includes predefined emoticons 96, such as ":-) " for a 
smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined 
emoticons are represented either as icons or as text, such as ";-)". As shown in FIG. 6, 
the window 80 includes a sample group of emoticon icons 96. The sender inserts an 
emoticon into a text message at the location of the cursor 102 by clicking on one of the 
emoticon icons 100. The sender may also type in the desired emoticon as text. 
Emoticon icons 96 save the sender from needing to type three keys, such as ":" and "-" 
and ")" for a smile. The icons 96 may be either a picture of, say, a winking eye or a icon 
representation of the characters ";-)" 100, or other information indicating to the sender 
that clicking on that emoticon icon will insert the associated emotion 103 into the text at 
the location of the cursor 102" (Column 8, lines 35-50) and "The sender can associate 
typed words with an emoticon by underlining, coloring, highlighting, or by any other 
means. For example, the method may comprise providing the sender an option to 
assign a color to the at least typed one word such that the chosen emoticon begins to 
be presented by the animated entity to the recipient at the first typed word with the 
assigned color and the chosen emoticon presentation by the animated entity ends at the 
last typed word with the assigned color. In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
continue the smile through the word "up". The method comprises receiving the indicated 
duration of the emotion and presenting the chosen duration of the emotion as the 
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animated entity delivers the message. As mentioned above, the highlighting can occur 
through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

The examiner further notes that Figure 6 of Ostermann displays a selectable 
area 96 for users to insert emoticons directly via a mouse into a currently typed 
message. Moreover, Ostermann explicitly states that the style markings of an adjacent 
character are applied to an emoticon (See "the sender an option to assign a color to the 
at least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color") 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 8, Hyon teaches a computer readable recording medium 
comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Paragraphs 23, 25, and 44, Figure 2); 

B) said registered image stored in a storage portion different from a storage portion of 
storing the predetermined kind of character (Paragraphs 23 and 44); 

C) at least one predetermined kind of character that is to be displaved directiv adjacent 
to said registered image with respect to display data containing a character code, text 
attribute data, and said image code (Paragraphs 21 and 41-42); 

F) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42); 
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The examiner notes tliat Hyon teaclies "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing by a user" as "A 

storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. Tlie storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23), "storage 18 stores oriental or occidental emoticons according to the 
cultural area of a user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a 
mobile terminal so that a user easily select an intended emoticon in the present 
invention. In another embodiment, the user can add hieroglyphics to basic emoticons 
stored by a manufacturer. The hieroglyphics can be stored in three ways: the user 
directly stores them, receives them from a base station and the stores them, or 
downloads them from the Internet. The user can change and edit the emoticons stored 
in the mobile terminal" (Paragraph 44). The examiner further notes that it is clear that 
the emoticons which are edited, changed, and user-generated must be stored in the 
RAM of storage 1 8 since items stored in RAM can be written to whereas items stored in 
ROM cannot be written (i.e. are Read-Only). The examiner further notes that items 
which are clearly not edited/changed/manipulated (i.e. text characters) would be stored 
in ROM of storage 1 8. Moreover, the examiner further wishes to state that it is well 
known that text characters from a particular font are stored in ROM and not RAM (see 
cited reference of Hersum (see column 4, lines 55-67 for reference). The examiner 
further notes that Hyon teaches "said registered image stored In a storage portion 
different from a storage portion of storing the predetermined kind of character" 
as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random 
Access Memory) for storing programs and data, and a voice memory. The storage 1 8 
stores an operation program for inputting emoticons and a plurality of emoticons in the 
form of a bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
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(Paragraph 23) and "In another embodiment, the user can add hieroglyphics to basic 
emoticons stored by a manufacturer. The hieroglyphics can be stored in three ways: 
the user directly stores them, receives them from a base station, and the stores them, or 
downloads them from the Internet" (Paragraph 44). The examiner further notes that 
Hyon teaches " at least one predetermined kind of character that is to be displayed 
directly adjacent to said registered image with respect to display data containing 
a character code, text attribute data, and said image code" as "a display 16 outputs 
display data and text messages generated in the mobile terminal. An LCD (Liquid 
Crystal Display) can be used as the display 16" (Paragraph 21), "If the user selects 
"smile" in the display 6 by manipulating a left/right directional key, emoticons that belong 
to the group "smile" are displayed in a display 7. When the user chooses one of the 
"smile" emoticons " . " in the display 7, the selected emoticon is stored as part of the 
short message as shown in a display 8. Then, the user can input a text including typical 
characters, special characters, or emoticons within the range of a transmittable SMS 
message, by changing input modes" (Paragraph 41), and "After an intended text is 
completed, the user instructs the mobile terminal to transmit the SMS message 
including the emoticon... the mobile terminal transmits the stored message" (Paragraph 
42). The examiner further notes that Hyon teaches "displaying said predetermined 
kind of character and said registered image simultaneously based on said 
transformed registered image" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
used as the display 16" (Paragraph 21), "If the user selects "smile" in the display 6 by 
manipulating a left/right directional key, emoticons that belong to the group "smile" are 
displayed in a display 7. When the user chooses one of the "smile" emoticons " . " in the 
display 7, the selected emoticon is stored as part of the short message as shown in a 
display 8. Then, the user can input a text including typical characters, special 
characters, or emoticons within the range of a transmittable SMS message, by changing 
input modes" (Paragraph 41), and "After an intended text is completed, the user 
instructs the mobile terminal to transmit the SMS message including the emoticon... the 
mobile terminal transmits the stored message" (Paragraph 42). The examiner further 
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notes that it is common knowledge that when text messages are received via a device, 
the entire message is displayed at once. 
Hyon does not explicitly teach: 

D) said text attribute data containing a size mark to be added to each said at least one 
directly adjacent character : 

E) transforming said registered image to be displayed according to said text attribute 
data; 

H) by determining whether or not a size of said at least one directly adjacent character 
and a size of the registered image are substantially the same, when the size of at least 
one directly adjacent character and the size of the registered image are not substantially 
the same, said display control portion makes the size of the at least one directly 
adjacent character and the size of the registered image substantially the same by 
scaling up/down the size of the registered image based on the text attribute data; 
J) causing said display output portion to display a text containing said at least one 
directly adjacent character and the registered image scaled to substantially the same 
size . 

Baker, however, teaches "said text attribute data containing a size marit to 
be added to each said at least one directly adjacent character " as "in order to 

accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), 
"transforming said registered image to be displayed according to said text 
attribute data" as "in order to accommodate the use of different size fonts in the 
mailbox display, means for scaling the size of the icon graphics are also provided... at 
least one image for each icon is stored, the icon most closely matching the point size of 
the font is chosen and then scaled as needed to better match the font point size" 
(Column 8, lines 26-40), "by determining whether or not a size of said at least one 
directly adjacent character and a size of the registered image are substantially the 
same, when the size of at least one directly adjacent character and the size of the 
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registered image are not substantially the same, said display control portion 
makes the size of the at least one directly adjacent character and the size of the 
registered image substantially the same by scaling up/down the size of the 
registered image based on the text attribute data" as "in order to accommodate the 

use of different size fonts in tlie mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and "causing said display 
output portion to display a text containing said at least one directly adjacent 
character and the registered image scaled to substantially the same size " as "in 
order to accommodate the use of different size fonts in the mailbox display, means for 
scaling the size of the icon graphics are also provided... at least one image for each icon 
is stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
D) said text attribute data containing a stvie mark to be added to each said at least one 
directiv adjacent character : 

G) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line; 

I) modifying said registered image to include image data for a mark resembling said 
style make attribute data of said at least one directiv adjacent character : 
J) and including the mark resembling the style mark attribute data in a position that is 
substantially continuous with the placement of the style mark in said at least one directly 
adjacent character . 
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Ostermann, however, teaches "said text attribute data containing a style 
mark to be added to each said at least one directly adjacent character " as "The 
sender may also insert emoticons 103 into the text of the message. The system 
includes predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" 
for an eye wink, and so forth. The predefined emoticons are represented either as icons 
or as text, such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of 
emoticon icons 96. The sender inserts an emoticon into a text message at the location 
of the cursor 102 by clicking on one of the emoticon icons 100. The sender may also 
type in the desired emoticon as text. Emoticon icons 96 save the sender from needing 
to type three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a 
picture of, say, a winking eye or a icon representation of the characters ";-)" 100, or 
other information indicating to the sender that clicking on that emoticon icon will insert 
the associated emotion 103 into the text at the location of the cursor 102" (Column 8, 
lines 35-50) and "The sender can associate typed words with an emoticon by 
underlining, coloring, highlighting, or by any other means. For example, the method 
may comprise providing the sender an option to assign a color to the at least typed one 
word such that the chosen emoticon begins to be presented by the animated entity to 
the recipient at the first typed word with the assigned color and the chosen emoticon 
presentation by the animated entity ends at the last typed word with the assigned color. 
In this case, in a sentence such as "Hi John, :-) are you pleased that the stock market is 
up?" the underlining represents the highlighting wherein the sender chooses to begin 
the smile at the beginning of the word "are" and to continue the smile through the word 
"up". The method comprises receiving the indicated duration of the emotion and 
presenting the chosen duration of the emotion as the animated entity delivers the 
message. As mentioned above, the highlighting can occur through coloring words, 
underlining words, or some other means of presenting the emotion" (Column 1 1 , lines 
43-61 ), "so that said registered image is sandwiched between two parts of said 
text of said predetermined kind of character in a line" as "The sender may also 
insert emoticons 103 into the text of the message. The system includes predefined 
emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye wink, and 
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so forth. The predefined emoticons are represented either as icons or as text, such as 
";-)". As shown in FIG. 6, the window 80 includes a sample group of emoticon icons 96. 
The sender inserts an emoticon into a text message at the location of the cursor 102 by 
clicking on one of the emoticon icons 100. The sender may also type in the desired 
emoticon as text. Emoticon icons 96 save the sender from needing to type three keys, 
such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture of, say, a 
winking eye or a icon representation of the characters ";-)" 100, or other information 
indicating to the sender that clicking on that emoticon icon will insert the associated 
emotion 103 into the text at the location of the cursor 102" (Column 8, lines 35-50) and 
"The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
the sender an option to assign a color to the at least typed one word such that the 
chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color. In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61), " modifying 
said registered image to include image data for a marl< resembling said style 
make attribute data of said at least one directly adjacent character " as "The sender 
may also insert emoticons 103 into the text of the message. The system includes 
predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye 
wink, and so forth. The predefined emoticons are represented either as icons or as text, 
such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of emoticon 
icons 96. The sender inserts an emoticon into a text message at the location of the 
cursor 102 by clicking on one of the emoticon icons 100. The sender may also type in 
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the desired emoticon as text. Emoticon icons 96 save the sender from needing to type 
three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture 
of, say, a winking eye or a icon representation of the characters ";-)" 100, or other 
information indicating to the sender that clicking on that emoticon icon will insert the 
associated emotion 103 into the text at the location of the cursor 102" (Column 8, lines 
35-50) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color. In this case, in 
a sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ), and " and 
including the mark resembling the style mark attribute data in a position that is 
substantially continuous with the placement of the style mark in said at least one 
directly adjacent character " as "The sender may also insert emoticons 103 into the 
text of the message. The system includes predefined emoticons 96, such as ":-) " for a 
smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined 
emoticons are represented either as icons or as text, such as ";-)". As shown in FIG. 6, 
the window 80 includes a sample group of emoticon icons 96. The sender inserts an 
emoticon into a text message at the location of the cursor 102 by clicking on one of the 
emoticon icons 100. The sender may also type in the desired emoticon as text. 
Emoticon icons 96 save the sender from needing to type three keys, such as ":" and "-" 
and ")" for a smile. The icons 96 may be either a picture of, say, a winking eye or a icon 
representation of the characters ";-)" 100, or other information indicating to the sender 
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that clicking on that emotlcon icon will insert the associated emotion 103 into the text at 
the location of the cursor 102" (Column 8, lines 35-50) and "The sender can associate 
typed words with an emotlcon by underlining, coloring, highlighting, or by any other 
means. For example, the method may comprise providing the sender an option to 
assign a color to the at least typed one word such that the chosen emoticon begins to 
be presented by the animated entity to the recipient at the first typed word with the 
assigned color and the chosen emoticon presentation by the animated entity ends at the 
last typed word with the assigned color. In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
continue the smile through the word "up". The method comprises receiving the indicated 
duration of the emotion and presenting the chosen duration of the emotion as the 
animated entity delivers the message. As mentioned above, the highlighting can occur 
through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

The examiner further notes that Figure 6 of Ostermann displays a selectable 
area 96 for users to insert emoticons directly via a mouse into a currently typed 
message. Moreover, Ostermann explicitly states that the style markings of an adjacent 
character are applied to an emoticon (See "the sender an option to assign a color to the 
at least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color") 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 
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Regarding claim 1 1 , Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said style mark is an underline said display control portion causes said 
display output portion to display the text containing said at least one directly adjacent 
character and the registered image including the mark resembling the underline in a 
position that is substantially continuous with the placement of the underline in said at 
least one directly adjacent character . 

Ostermann, however, teaches " wherein said style mark is an underline said 
display control portion causes said display output portion to display the text 
containing said at least one directly adjacent character and the registered image 
including the mark resembling the underline in a position that is substantiallv 
continuous with the placement of the underline in said at least one directly 
adjacent character " as "The sender may also insert emoticons 103 into the text of the 
message. The system includes predefined emoticons 96, such as ":-) " for a smile, "::-)" 
for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)"■ As shown in FIG. 6, the window 80 
includes a sample group of emoticon icons 96. The sender inserts an emoticon into a 
text message at the location of the cursor 102 by clicking on one of the emoticon icons 
100. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" 100, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color. In this case, in a sentence such as "Hi John, :-) are you pleased 
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that the stock market is up?" the underlining represents the highlighting wherein the 
sender chooses to begin the smile at the beginning of the word "are" and to continue 
the smile through the word "up". The method comprises receiving the indicated duration 
of the emotion and presenting the chosen duration of the emotion as the animated entity 
delivers the message. As mentioned above, the highlighting can occur through coloring 
words, underlining words, or some other means of presenting the emotion" (Column 1 1 , 
lines 43-61). 

The examiner further notes that Figure 6 of Ostermann displays a selectable 
area 96 for users to insert emoticons directly via a mouse into a currently typed 
message. Moreover, Ostermann explicitly states that the style markings of an adjacent 
character are applied to an emoticon (See "The sender can associate typed words with 
an emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color"). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 13, Hyon further teaches a display device comprising: 

A) wherein the first storage portion is a ROM (Paragraph 23); and 

B) the second storage portion is a RAM (Paragraphs 23 and 44). 

The examiner notes that Hyon teaches "wherein the first storage portion is a 
ROM" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
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emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23). The examiner further notes that items which 
are clearly not edited/changed/manipulated (i.e. text characters) would be stored in 
ROM of storage 18. Moreover, the examiner further wishes to state that it is well known 
that text characters from a particular font are stored In ROM and not RAM (see cited 
reference of Hersum (see column 4, lines 55-67 for reference). The examiner further 
notes that Hyon teaches "the second storage portion is a RAIVI" as "A storage 18 Is 
comprised of a ROM (Read Only Memory) and a RAM (Random Access Memory) for 
storing programs and data, and a voice memory. The storage 18 stores an operation 
program for Inputting emoticons and a plurality of emoticons in the form of a bit map 
according to the embodiment of the present invention. The plurality of emoticons are 
stored by groups In the storage 18 to facilitate selection of emoticons" (Paragraph 23) 
and "A plurality of emoticons are pre-stored in a mobile terminal so that a user easily 
select an intended emoticon in the present invention. In another embodiment, the user 
can add hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics 
can be stored in three ways: the user directly stores them, receives them from a base 
station and the stores them, or downloads them from the Internet. The user can change 
and edit the emoticons stored In the mobile terminal" (Paragraph 44). The examiner 
further notes that it is clear that the emoticons which are edited, changed, and user- 
generated must be stored in the RAM of storage 18 since items stored in RAM can be 
written to whereas items stored in ROM cannot be written (i.e. are Read-Only). 

Regarding claims 14-17, Hyon and Baker do not explicitly teach a display 
device, display method, viewer program, and computer-readable recording medium 
comprising: 

A) wherein the text containing the predetermined kind of character and the registered 
image is used in a Hyper Text Markup Language document. 

Ostermann, however, teaches "wherein the text containing the 
predetermined kind of character and the registered image is used in a Hyper Text 
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Markup Language document" as "FIG. 4(a) illustrates a high-bandwidth architecture 
60 associated with the embodiments of the invention. The system 60 delivers a hyper- 
text mark-up language (HTML) page through the Internet 62 (connected to a web 
server, not shown but embodied in the Internet 62) to a client application 64. The HTML 
page (shown by way of example in FIG. 6) enables the sender to create a multi-media 
message. The client application may be, for example, a web browser such as 
Microsoft's Internet Explorer. RTM.. Other client applications include e-mail and instant 
messaging clients. The sender creates the multi-media message using the HTML page" 
(Column 4, lines 53-63). 

The examiner notes that it is clear that Ostermann's method can use various 
computer font techniques to apply to sentences with emoticons, such as underlines and 
highlighters. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

8. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hyon 
(U.S. PGPUB 2002/0077135) in view of Baker (U.S. Patent 6,546,417), and further in 
view of Ostermann et aL (U.S. Patent 6,990,452) as applied to claims 1 , 4, 6-8, 1 1 , and 
13-17 above, and further in view of Prouix et aL (U.S. PGPUB 2002/0124019). 

9. Regarding claim 1 2, Hyon and Baker do not explicitly teach a display device 
comprising: 

B) said display control portion causes said display output portion to display the text 
containing said at least one directly adjacent character and the registered image 
including the mark in a position that is substantially continuous with the placement of the 
line in said at least one directly adjacent character . 

Ostermann, however, teaches " said display control portion causes said 
display output portion to display the text containing said at least one directly 
adiacent character and the registered image including the mark in a position that 
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is substantially continuous with the placement of the line in said at least one 
directly adjacent character " as "The sender may also insert emoticons 103 into the 
text of the message. The system includes predefined emoticons 96, such as ":-) " for a 
smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined 

emoticons are represented either as icons or as text, such as ";-)". As shown in FIG. 6, 
the window 80 includes a sample group of emoticon icons 96. The sender inserts an 
emoticon into a text message at the location of the cursor 102 by clicking on one of the 
emoticon icons 100. The sender may also type in the desired emoticon as text. 
Emoticon icons 96 save the sender from needing to type three keys, such as ":" and "-" 
and ")" for a smile. The icons 96 may be either a picture of, say, a winking eye or a icon 
representation of the characters ";-)" 100, or other information indicating to the sender 
that clicking on that emoticon icon will insert the associated emotion 103 into the text at 
the location of the cursor 102" (Column 8, lines 35-50) and "The sender can associate 
typed words with an emoticon by underlining, coloring, highlighting, or by any other 
means. For example, the method may comprise providing the sender an option to 
assign a color to the at least typed one word such that the chosen emoticon begins to 
be presented by the animated entity to the recipient at the first typed word with the 
assigned color and the chosen emoticon presentation by the animated entity ends at the 
last typed word with the assigned color. In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
continue the smile through the word "up". The method comprises receiving the indicated 
duration of the emotion and presenting the chosen duration of the emotion as the 
animated entity delivers the message. As mentioned above, the highlighting can occur 
through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

The examiner further notes that Figure 6 of Ostermann displays a selectable 
area 96 for users to insert emoticons directly via a mouse into a currently typed 
message. Moreover, Ostermann explicitly states that the style markings of an adjacent 
character are applied to an emoticon (See "The sender can associate typed words with 
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an emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color"). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Hyon, Baker, and Ostermann do not explicitly teach: 

A) wherein said stvie mark is a cancel line : and 

B) resembling the cancel line . 

Prouix, however, teaches " wherein said style mark is a cancel line " and 
" resembling the cancel line " as "The font type represents information about the style 
of the font. One embodiment has seven font type options: nochange, textbody, bold, 
italic, headingi, heading2 and heading. The text attributes represent information about 
the style of the text. One embodiment has six text attribute options: bold, italic, 
underline, striketh rough, outline and shadow" (Paragraph 49). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Prouix's would have allowed Hyon's, Baker's, and Ostermann's to provide a method 
to allow the ability to insert multiple style settings to documents, as noted by Prouix 
(Paragraph 21). 

Response to Arguments 
1 0. Applicant's arguments filed on 08/05/2010 have been fully considered but they 
are not persuasive. 

Applicants argue on page 11 that "Images, unlike symbols, are not 
predefined, and thus are not provided with attributes as in text". However, the 



Application/Control Number: 10/516,881 
Art Unit: 2168 



Page 44 



examiner wislies to refer to Columns 8 and 1 1 of Ostermann which state "The sender 
may also insert emoticons 103 into the text of the message. The system includes 
predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye 
wink, and so forth. The predefined emoticons are represented either as icons or as text, 
such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of emoticon 
icons 96. The sender inserts an emoticon into a text message at the location of the 
cursor 102 by clicking on one of the emoticon icons 100. The sender may also type in 
the desired emoticon as text. Emoticon icons 96 save the sender from needing to type 
three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture 
of, say, a winking eye or a icon representation of the characters ";-)" 100, or other 
information indicating to the sender that clicking on that emoticon icon will insert the 
associated emotion 103 into the text at the location of the cursor 102" (Column 8, lines 
35-50) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color. In this case, in 
a sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emotion and presenting the 
chosen duration of the emotion as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ). The 
examiner further notes that Figure 6 of Ostermann displays a selectable area 96 for 
users to insert emoticons directly via a mouse into a currently typed message. 
Moreover, Ostermann explicitly states that the style markings of an adjacent character 
are applied to an emoticon (See "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example. 
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the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color"). 

Applicants argue on page 1 1 that "the attributes of height and width for 
images do not coincide with the attribute size for text". However, the examiner 
wishes to refer to Column 8 of Baker which states "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40). The examiner further wishes 
to state that it is clear that Baker scales images to the size of a font of an adjacent text 
character. 

Applicants argue on page 1 1 that "the cited prior art, either alone or in 
combination, fail to disclose at least the feature recited in claim 1 as amended of, 
"said display control portion. ..adjacent character". However, the examiner wishes 
to state that it is clear that Baker scales the size of an image to the size of adjacent 
characters while Ostermann applies style marks to adjacent characters to a selectable 
emoticon image. 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

U.S. Patent 6,987,991 by Nelson on 17 January 2006. The subject matter 
disclosed therein is pertinent to that of claims 1, 4, 6-8, and 11-17 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. PGPUB 2002/0120653 by Kraft et al. on 29 August 2002. The subject 
matter disclosed therein is pertinent to that of claims 1 , 4, 6-8, and 11-17 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 
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U.S. Patent 6,584,479 by Chang et al. on 24 June 2003. The subject matter 
disclosed therein is pertinent to that of claims 1 , 4, 6-8, and 11-17 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. Patent 6,456,305 by Qureshi et al. on 24 September 2002. The subject 
matter disclosed therein is pertinent to that of claims 1, 4, 6-8, and 11-17 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. Patent 5,301,106 by Hersum on 05 April 1994. The subject matter 
disclosed therein is pertinent to that of claims 1 , 4, 6-8, and 11-17 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

Article entitled "WD2000: Underlining Text Causes Picture to be Underlined" by 
Microsoft, dated 06 December 2000. The subject matter disclosed therein is pertinent 
to that of claims 1 , 4, 6-8, and 11-17 (e.g., methods automatically scale graphics into a 
text sentence that is sent to another user). 

Contact Information 
1 2. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272- 
2731 . The examiner can normally be reached on Monday to Friday 8:20 am - 4:40 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Tim Vo can be reached (571 ) 272-3642. The fax number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.qov . Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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